Color cosmetics have been applied to beautify the human body since antiquity. But a recent patent review demonstrates that technology is creating new ways to create and apply cosmetics. The portable 3D makeup printer is on track to revolutionize and personalize beauty care.¹ The 3D printing of color palettes is here and it is the start of a bigger trend. The ubiquity of digital cameras makes it possible to augment reality by using captured digital images. For example, it is now possible to try on clothes in a virtual dressing room, using an accurate image of one’s own body, or to apply virtual makeup to an image of one’s own face.^2,3 It is notable that this patent issued on Halloween! Going beyond mere image processing, one patent application advanced the notion that your makeup and style could be computationally analyzed to reveal your true character.⁴ That application was rejected on the grounds that the description was merely an abstract idea that had been previously disclosed in the inventor’s own blog. Thank goodness for the difference between science fiction and science! But, with time, that difference could be eroded. In a world in which consumers will be scanned, and clothes and shoes will be robotically custom-made, techniques arising from facial recognition will accurately present a 3D model of a customer’s face, allowing the choice of colors and applies the correct makeup to match skin tone.⁵ If the client has acne, rosacea or another condition that might mask the user’s skin tone, pictures can be taken of the skin directly behind the shell of the ear to get an accurate measure of skin tone.⁶


Accurately and reproducibly decorating one’s face can be difficult and time-consuming. To make it easier, one patent application provides adhesive colored plastic shapes to stick to the eyelids as eyeshadow, the lips as lipstick and the cheeks as makeup.⁷ Unfortunately, the application was abandoned following its rejection on the grounds that it had been anticipated by color-adaptable adhesive bandages⁸ and it was obvious from prior art for colored patches that could be applied to non-hairy skin.⁹ L’Oréal suggests ink-jet printing the makeup patterns on a plastic sheet and transferring the perfect images to one’s lips or other part of the facial anatomy as desired. The makeup pigments and dyes are loaded into standard printer cartridges. After applying the color to the face, a film of Eastman Sulfopolyester AQ resin, BASF’s Ultrahold Strong (acrylates/t-butylacrylamide copolymer or Elnett hairspray is sprayed over the color to protect it from transfer, wear, or wash-off.^10,11,12 L’Oréal is offering consumers the convenience of professionally-applied virtual makeup.¹³


AR for a Flawless Look

For consumers unskilled in makeup application, facial recognition assistance will be available. Once the face is expertly made-up, a 3D image of the face is captured and masks are printed on thin films of peelable polymers, like polylactic acid. When it come time to apply makeup, the masks are applied to the face and peeled off to give a perfect finish every time.^{14,15,16,17,18}

With their home appliance markets maturing, Japanese home electronics makers have targeted beauty. Panasonic has introduced a “future mirror” that allows a person to apply virtual makeup until they get the look they desire; then one touch of the screen texts or emails expert directions for achieving the look for real!¹⁹ The smart mirror can even highlight facial flaws and print out a makeup sheet to expertly cover the flaws, like a very precise temporary tattoo.^20,21 The mirror can even detect sub-surface flaws before they erupt and suggest appropriate makeup ideas for pre-treatment. Panasonic expects to launch its Snow Beauty Mirror in 2020. Using technology originally developed for organic light emitting diodes, upon completing the analysis of a face, at the touch of the screen a special printer tailor-makes an ultrathin makeup-coated patch that can be applied to the face. The barely-perceptible patch resists removal until it is washed with water.²²

Professors Kaoru Arakawa and Fumihiko Tsukiyama, of Meiji University and Kosé Corporation, disclose a face image processing method that can analyze a client’s face, identify flecks, pores and wrinkles and then allow the client to adjust the image to the preferred style corresponding to a desired age or age-group to guide the selection and application of makeup.²³

How long it will be before we use remote-controlled 3D printers to apply makeup precisely on the face online anywhere in the world via an internet connection. Will “future mirrors” become standard equipment installed behind automobile sunshades? Will compacts be replaced by printer cartridges?

One constant in the cosmetic industry is the change of styles at the dictates of fashion. Panasonic took style changes into consideration in its makeup trend-analyzing apparatus and method. In this, makeup simulators present a range of makeups as advanced options for selection. The selections are processed, and makeup trends are analyzed and predicted.²⁴

When will robotic makeup artists appear on the scene? 


About the Authors
Following a 25-year career with ICI, Unilever, and BF Goodrich, Bob Lochhead joined academia and served as professor, chair and director of the School of Polymers and High Performance Materials at the University  of Southern Mississippi. Bob has served twice as the president of the SCC, and also as president of the International Association of Formulation Chemists. He is a fellow of the SCC and a Fellow of the Royal Society of Chemistry. He is a recipient of the SCC’s Maison G. DeNavarre Medal Award, the SCC’S Merit Award, the AIChE’s Chemical Pioneer Award, and the Society of Plastics Engineers’ Education Award. Bob will receive the SCC’s Lifetime Service Award this month. 
 
Margaret Lochhead's initial occupation was biochemist/physiologist/ statistician and for the latter part of her career she taught developmental mathematics at the University of Southern Mississippi. Margaret is a member of the SCC.
 
 


References
1. The Mink, the World’s First-Ever 3D Portable Makeup Printer, Is Officially Available for Pre-Order, https://www.allure.com/story/mink-3d-makeup-printer
2. Siddiqui, Matheen; Wnuk, Kamil; Augmented reality content rendering via albedo models, systems and methods, U.S. Patent 9,805,510, Oct. 31, 2017. Applicant Nant Holdings IP, LLC. Culver City, CA. 
3. Siddiqui, Matheen; Wnuk, Kamil; Augmented reality content rendering via albedo models, systems and methods, U.S. Patent 10,192,365, Jan 29, 2019. Assigned to Nant Holdings IP, LLC. Culver City, CA.
4. Mychals, Brandy; Personality profiling system and methods for use in all types of communications and interactions, U.S. Patent Application 20130344464, Dec. 26, 2013.
5. Tran, Bao; Augmented Reality System, U.S. Patent Application 20180253906, Sept. 6, 2018.
6. Tran, Bao; AR System, U.S. Patent Application 20190228570, July 25, 2019.
7. Meriaux, C; Facial cosmetic devices and systems, U.S. Patent Application 2006/0013844, Jan 19, 2006.
8. Harris, F.E., Pavitt, W.H.; Color adaptable bandage, U.S. Patent 4,161,176, July 17, 1979.
9. Muchin, J.D.; Dermatological Patch, U.S. Patent 6,241,998, Jun 5, 2001, Assigned to Accutek International.
10. Samain, Henri; Giron, Franck; Gevrey, Chrystele; Transfer device for making up keratin materials, U.S. Patent Application 20160324298, Nov. 10, 2016. Applicant: L’Oréal
11. Samain, Henri; Giron, Franck; Transfer device for making up keratin materials, U.S. Patent Application 20160316891, Nov. 3, 2016. Applicant L’Oréal
12. Samain, Henri; Giron, Franck; Transfer device for making up keratin materials, U.S. Patent Application 20160316890, Nov. 3, 2016. Applicant L’Oréal.
13. L’Oréal Is bringing the makeup counter experience into your home with AR and livestreaming, https://www.adweek.com/brand-marketing/loreal-is-bringing-the-makeup-counter-experience-into-your-home-with-ar-and-livestreaming/
14. Nishi, Chie; Takeshita, Sachiko; Asai, Rieko; Taoka, Hiroki; Shinoda, Masayo; Makeup part generating apparatus, makeup part utilizing apparatus, makeup part generating method, makeup part utilizing method, non-transitory computer-readable recording medium storing makeup part generating program, and non-transitory computer-readable recording medium storing makeup part utilizing program, U.S. Patent Application 20180268572, Sept. 20, 2018, Assigned to Panasonic Intellectual Property Management Co., LTD. Osaka.
15. Ajiki, Kaori; Asai, Rieko; Yamaguchi, Yasushi; Yamanashi, Tomofumi; Muta, Aoi; Nishi, Chie; Makeup support device, makeup support method, and makeup support program, U.S. Patent Application 20160022014, Jan 28, 2016, Assigned to Panasonic Intellectual Property Management Co., LTD. Osaka.
16. Ajiki, Kaori; Asai, Rieko; Yamaguchi, Yasushi; Yamanashi, Tomofumi; Muta, Aoi; Nishi, Chie; Makeup support device, makeup support method, and makeup support program, U.S. Patent Application 20190150595, May 23, 2019, Assigned to Panasonic Intellectual Property Management Co., LTD. Osaka.
17. Shinoda; Masayo, Image processing apparatus and method for processing image, U.S. Patent Application 20190098149, March 28, 2019, Assigned to Panasonic Intellectual Property Management Co., LTD. Osaka.
18. Yamanashi, Tomofumi; Asai, Rieko; Muta, Aoi; Nishi, Chie; Ajiki, Kaori; Makeup application assistance device, makeup application assistance method, and makeup application assistance program, U.S. Patent 9,681,736,
June 20, 2017, Assigned to Panasonic Intellectual Property Management Co., LTD. Osaka
19. The Future Mirror Knows Your Beauty Secrets #PanasonicCES 2016. https://www.youtube.com/watch?v=pJubKcmsFQ4
20. Panasonic’s smart mirror finds your flaws, prints makeup to fix them, https://www.engadget.com/2016/10/04/panasonic-concept-mirror-makeup-printer/
21. Panasonic Has Combined Various Technologies to Revolutionize Makeup. https://www.youtube.com/watch?v=vzmK8ojRzgU
22. Shusuke Murai, Japan Times, Jan. 8, 2017, https://www.japantimes.co.jp/news/2017/01/08/business/tech/next-generation-mirror-analyzes-faces-prints-customized-makeup-sheets/#.XZZaZEZKg2w
23. Arakawa, K. and Tsukiyama, F., Face image processing system, face image processing method, and face image processing program, U.S. Patent Application 2019/0244334, August 8, 2019. Assigned to Meiji University and Kose Corporation.
24. Takei, Ichiro; Makeup trend analyzing apparatus, makeup trend analyzing method, and non-transitory computer-readable recording medium storing, makeup trend analyzing program, U.S. Patent Application 20180276453, September 27, 2018, Assigned to Panasonic Intellectual Property Management Co., LTD. Osaka.